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The Center for Cooperative Research with Schools 
(CReHS) contracted to develop a management iviforiation system (MIS) 
for the National Regional Resources Center of Pennsylvania (NRRCP) . 
The system was originally intended to tcansforii NRRCP data into 
management — oriented information to support decision-making within 
the agency. After a year's work, however, the NRRCP was pressured by 
its funding sources for accountability data and thus requested the 
fast delivery of a completely different MIS stressing project 
monitoring, cost analysis, and benefit assessment. The resulting MIS 
generated voluminous reports, but little useful information, and the 
cost in staff tine was high. CReHS* s experience has led to the 
following four conclusions. First, the demand for accountability 
generally leads to detailed but incomprehensible reports. Second, a 
useful MIS cannot be developed until the agency first clearly defines 
its goals. Third, the expected benefits of new management tools are 
usually unrealistic. Finally, information and management consultants 
should be wary of taking the practitioner's word for what he needs. 
An objective need assessment is nore likely to result in the 
production of a useful, relevant management system. (PB) 
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COMIN/5 OF AQi: OF THE FAMILY CReWS: 
A BALU'f HI THREE ACTS 



By 

vJohn L. Hayman, Or. 
and 

Barry A. Clemson 

Center for Cooperative Research with Schools 
The Pennsylvania State University 

1. THE BALLET 

Once upon a time In the State of Pennsylvania and the County of Centre, 
there lived a quiet family called CReUS (Center for Cooperative Research with 
Schools). UUh all oood conscience and high Inspiration, the family CReWS 
devoted Itself to spreading systems approaches throughout the land, and soon 
It was proselytizing and practicing the ancient art of grantsmanshlp. 

It came to pass that the; men of CReWS encountered a group with need 
(and with money), and soon a deal was made, that to the National Regional 
Resources Center of Pennsylvania (NRRC/P) would be delivered an elixir— 
ar. MIS to solve Its problems. 

The story which follows, dear reader. Is of the attempt to make that 
elixir, and of the coming of age of the family CReWS which accompanied It. 
It is a ballet In three acts: a stately minuet, an explosive flamenco, and 
a hectic Jitterbug, 

Before this saga goes on, you must first be told that IIRRC/P Is a state- 
wide group concerned with the education of handicapped children. It both 
develops procedures for this work and disseminates the procedures to public 
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school systems and their people. In the beginning, NRRC/P had multfttides of 
data. The elixir was to change these base data Into the gold of management- 
oriented information. 

THE MINUET 

The minuet, stately and dignified* began with a request to CReWS to 
design and implement a management Information system (MIS) to support adminis- 
trative decision making. The MIS was to include evaluation, accountability, 
and cost data. The capstone of the system was to be a cost-benefit capability 
—at least to the extent possible given the current state of the art. 
Production of Background Data 

. The first task was to study the agency. An intensive field study, which 
included interviews, site-visits, wrkshops with agency personnel, examination 
of forms, and analysis of the agency's reporting requiretrients, v/as conducted. 

The Interviews uere with the agency's top personnel. They were open- 
ended discussions v/hich covered these points: 

1. Main characteristics of each administrative unit. 

2. Boundaries of each unit. 

3. Major activities of each unit. 

4. Future opportunities. 

5. Major stumbling blocks or problems. 

$. Heeds of the target children and schools. 

7. Major objectives of each unit. 

8. Major resources of each unit. 

9. Major strategies utilized to achieve the objectives. 
10. Evaluation efforts currently under way. 
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11. Issues or areas of the unit's operations where more Information Is 
needed. 

12. Records maintained. 

The Interviews were followed by site visits and observations of Instruc- 
tional personnel in 15 different schools. The site visits had mul^^lple 
purposes, including: 

' Observation of the various instructional personnel. 

' Observation of the record keeping and report generating required of 
the staff. 

The site visits Included classroom observations and conversations with teachers 
and aides, principals, parents, and school support personnel such as psycholo- 
gists and counselors. 

The interviews an4 site visits provided data for en analysis of the 
aj^ency with two primary concerns: 

1. The constraints and limitations on the agency. 

2. The set of strategies and procedures the agency Intended to 
operational ize in the field. 

The latter part of the analysis (the strategies and procedures) provided the 
basis for two models of staff activities— one each for teaching and for 
management staff. CReUS staff also conducted two workshops designed to 
elicit input from a broad cross-section of agency personnel. The first of 
these focused on clarifying evaluation concerns, the second on refining the 
model of staff activities. After these activities, CReWS staff haJ an in-depth 
understanding of the agency and its needs, and it began to design the output 
capability of the proposed MIS. The intention was to provide summary infor- 
mation, changes over time Information, and cost-ratios information. ' 
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Developfnent of thei Time A ccounting $yste<n 

The models of staff activttlos wore pressed into service inwedlately. 
One of NRRC/P's big unknowns was the pattern of time expenditure for the 
various staff types, A Time Accounting system, designed to document the 
actual time expenditure pattorn» was implemented during the first ycixr. The 
Time Accounting system was a straightfomrd process for the staff to log 
their time in each of a variety of activities. The data was then computer 
processed and summarized by staff type. 

The models of staff activities utilized two different taxonomic scheri^esi 
1) activities groupod by project goaliand 2) activities grouped by type of 
service. The "Teacher Time-Record form" utilized the "activities by type of 
service classification. These data permitted determination of the relative 
costc of the various services provided to children and school staff. They 
also provided a check on the extent to which the staff were in fact functioning 
in the manner that the project managers intended, i.e., the extent to which 
the instructional model was being implemented. 

The "Manager's Time-Record Form" utilized the "activities by project 
goal" classification. This data permitted determination of costs for each 
of the project goals as well as a baseline for patterns of management 
activities. 

The Time-Record data is collected on a form that has activities listed 
down the side and the days of the month across the top. The Teachers form 
has two pages and the (Managers form has five pages (see Appendix). At the 
end of each day, each staff member records his time in half -hour units for 
each activity engaged In. 
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The Time-Record foms am collected at the end of the month, keypunched, 
and run through a computer program which provides a variety of reports, 
Including breakdowns for each staff type by each administrative unit. The 
program generates monthly reports and cumulative reports for three-month 
periods. The reports were generated for each operational sub-unit of the 
organization, and v/are aggregated from these to the State as a whole. 

THE FLAMENCO 

Just at the point when CReWS staff felt it was evolving a workable and 
useful MIS design, the tempo abruptly changed. The explosive, improvlsional 
flamenco arrived when the agency director, in a flash of insight, said, 
"Here, this is what I want," and pointed to a manual for a different sort of 
MIS used In another educational agency. An accountability data system (sort 
of I) was needed and was needed immediately. The funding agencies were 
demanding it and it therefore had to be provided. The fact that a year's 
planning and developmental activity had gone into a different design was of 
no avail. 

Intended outcomes such as support for decision-making, utility to agency 
personnel, reports depicting changes over time, cost-ratio information and 
summary information v;ere displaced by the urgent need to have accountability 
data Immediately. The exten^^lve field work and planning were suddenly back- 
ground material rather than p'^imary planning data. A short period of acute 
shock followed as the staff trieJ to learn the new dance and to re-orient 
Its objectives and activities for the project. Year-at-a-time funding 
contributed to the susceptibility of CRoHS to the agency director's changing 
insights. 

ERIC 
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THE JIHERDUQ 

Thd third and U^t phase was a mad rush akin to an uninhibited Jitter- 
bug—a hectic, arm-flailing, full speed ahead Implementation of a system to 
generate accountability data. The major activities consisted of designing 
report formats, forms and procedures to collect the data, and computer 
routines to process the data. 

The procedure for Implementing the "accountability data novi" MIS was to 
separate It into a number of modules. These modules were In the general 
areas of project monitorirtg, cost accounting, and benefit assessment. 
Figure 1 provides an outline of the various modules as they were conceptual- 
ized during this stage. 



Project 
Monitoring 


Cost 
Analysis 


Benefit 
Assessment 


Product Monitoring 
Service Honltorlng 


Cost Accounting . 
Time Accounting 


Service Impacts 
Product Impacts 



Figure 1 



The overall strategy for designing the MIS shifted considerably at this 
point. During the earlier phase, the strategy was to focus on the utility y 
of information for decision making. Operationally, this meant that the 
final products were designed first, and the rest of the system emerged as 
staff worked backward from that intended end result. Now, with the pressing 



need to have something operational , the focus of attention shifted to the 
structure outlined in Figure 1. The» final objective was unknown and would 
be determined as it emerged from tho process of combining the pieces. 

CReWS work to date has focused on three of the modules (Product 
Monitoring, Time Accountins, and Co$;t Accounting). The functions of these 
are: 

Product Monitoring - For each jjpecial project the agency Is to carry 
out, a detailed plan (much like a PERT network) is prepared, This 
plan is revised monthly as a means of monitoring the progress of the 
project. 

Time Accounting - Each staff rnember records his time expenditure 
using a set of activities designed to include all of his routine 
duties. The entries are made daily to the nearest half hour.; Note 
that this is a continuation of the system developed in the Minuet 
phase. 

Cost Accounting - The bookkeeping procedures are designed to allow 

cost accounting by objective, by service types, and by goal as well 

as by traditional line item budget categories. 

NRRC/P has a large scale in-house data collection system that is 
particularly relevant to the service monitoring and service impacts modules. 
The data collected by this system are in the form of detailed histories for 
individual students. Thus, despite a massive data base, there is no infor- 
mation in a form that is illuminating for management decisions. 

At the present stage of development, several generalizations can be 
made about these modules: 

1. A great deal of data is collected. 
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2. Very detailed and voluminous reports are generated. 

3. There Is very little In the way of summary reports, or a capability 
for juxtaposition of data from two or more sources* or reports which 
show changes over time. 

4. A not- inconsiderable amount of the entire staff's time Is required 
to support the data collection effort. 

5. While the product at this point is undeniably useful to the 
agency, the elixir of information for management decision-making 
is yet to bo produced. The cHsis orientation, the departure 
from a long-range planning perspective, the failure to employ 

a systems perspective instead of Just systems analysis tools-- 
hll have contributed to a situation characterized by a multitude 
of data and little information. 

11. COMING OF AGE 

It is instructive to look back and reflect on the project to this point, 
on the changes of tempo and harmonics, and on the maturity this has produced 
within CReWS. In retrospect, a number of lessons seem clear. Further, 
these lessons seem common to many educationaV projects, and perhaps others 
can come of age by reflecting on them. 

DEMAIID FOR ACCOUNTABILITY 

The various publics of educational institutions are all demanding 
accountability data. In the case of the NRRC/P, this demand came from 
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the Bureau of Education for the Handicapped In Washington (the source of 
funds via the State), from the State Department of Special Education, from 
the cooperating school districts, and from parents. The cry for accounta- 
bility Is generally Interpreted as a demand for data which "proves" that 
the Institution Is at least making good progress toward Its proper goals. 
The usual result of attempting to provide accountability data, as with the 
NRRC/P' Project, Is to generate reports which are totally Incomprehensible 
by way of their voluminous variety of detail. 

UilDEFINEO VALUES AND GOALS 

Education has been (and perhaps always will be) characterized by vague 
and ambiguous values and goals. Without entering the debate over behavioral 
objectives, we simply note that the NRRC/P, like other educational agencies, 
has great difficulty In stating broad operational objectives for Itself. A 
closely related point Is the lack of explicitly stated global goals or value 
judgments for the agency. To the extent that such global ^oals and value 
Judgments are absent, a number of crucial difficulties^ In organizational 
planning and management arise: 

1. It Is correspondingly difficult to determine the alternative 
strategies or the future decisions facing the agency. 



'This section owes a great debt to Dr. Robert Mowltz, Director of The 
Institute for Public Administration, The Pennsylvania State University, and 
his weirk with Planning, Programming, Budgeting Systems (PPBSj at the State 
government level. See foj* Instance hU repbrt oh Th6 Pennsylvania PPBS, 
The Design and Implementation of PennsylvahlaVs Manning . Pregrammlng , 
Wgeting Syste 'tti institute efTubiic Adm^nistratlohy PSU, 1970. 
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It Is difficult to define the Information needs of the agency. 
Information Is needed to evaluate strategies, provide data for 
various types of decisions, and generate feedback on current 
operating characteristics. If the strategies and decisions are 
unknown, the needed Information Is necessarily unspeciflable. 
It Is difficult to Interpret the data that are collected. Inter- 
pretation of program data for decision-making always Implies a 
value judgment {I.e., higher reading scores are preferred to lower, 
more Independence is preferred to less, lower costs are preferred 
to higher, faster progress Is preferred to flower). In practice, 
we generally operate by gathering data relevant to many of our 
strong values. However, we seldom make these values explicit. We 
therefore have no explicit way of Interpreting the data, and we 
tend to become Involved In mere counting of unlnterpretable entitles. 

It can be argued that people do In fact Interpret their data 
on the basis of Implicit values. Note, however, that this makes 
the process vary subjective and not susceptible to a scientific 
reconciliation of differences, because the major premises (and 
therefore applicable values) may be different for different parties. 
There is another dimension to the problem of interpreting data, 
Given an explicit goal (i.e., a well-defined value statement 
specifying the preferred conditions), then operational objectives 
and sub-objectives become working hypotheses on the causal relation- 
ships in the system. In other words, given goal A and objectives 
X, y, and 2, the working hypothesis is obviously that achieving 
objectives x, y, and 2 will move toward goal A. 
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Educators (Including the present authors) have, for a variety of reasons 
generally failed to make e«pl1cU connections between program objectives and 
ove**all goals. In the absence of explicit hypotheses related to these 
connections, we will continue to be Ignorant about the ways program effec- 
tiveness can be Improved, no matter how much raw data we collect. 

MODERN MANAGEMENT TOOLS 

Educators are responding rapidly to a variety of forces tending toward 
utilization of modern management tools such as PERT, PPBS, HIS, and systems 
analysis. These manageir«nt methods were, by and large, pioneered by the 
Pentagon and popularized by Industry. They have been adopted to some extent 
by other branches of the federal government and some state governments. 

As with anything new, the expectations and the reality never match In 
totoi all too often, they are orders of magnitudes apart. Mildavsky, the 
most persistent and perspicacious critic of the whole approach, has long 
argued that a) It won't vjork, and b) If It did work, It would be worse .than 
what we do now. 

Given the pressures and the temper of our times, one can safely predict 
that at least some educational institutions will rush to adopt new methods 
that are Inappropriate to their needs. The new methods are tools— no more, 
no less. Many of them are unreasonably expensive for Installation in most 
situations. Furthemore, lacking experience with them as we do, there is no 
way to guarantee that they will perform as expected in an educational milieu 
We wiif do well te proceed with caution and a thoroughly scientific temper. 
Wlldavsky (1973) sucigests ten precautionary r^ules for those considering an 
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1. The rule of skepticism? no one knows how to do It. Unless the 
Idea Is to subsidize employment of social scientists, the burden 
of proof should be on the proposer. 

2. The rule of delay: If It works at all* It vwn't work soon. Be 
prepared to give It years. 

3. The rule of complexity: nothing complicated v/orks, When a new 
Information system contains more variables than, shall we say, the 
average age of the officials who are to use It, , . , the chances 
of failure are very high. 

4. The rule of thumb: If the data are thicker than your thumb . . . 
they are not likely to be comprehensible to anyone. 

5. The fifth rule Is to be like a child. Ask many questions*, be literal 
In appraising answers. Unless you understand precisely who will 

use each data bit, how often, at what cost, relevant to which 
decisions they are empowered to make, don't proceed. 

6. ' Sixth Is the rule of length and width ... The longer the sequence 

of steps, the wider the band of clientele, the less likely the In- 
. formation Is to be of use. 

7. The rule of anticipated anguish . . . s most of the things that 
can go wrong will.. ... If you do not have substantial reserves 
of money, men, and time to help repair breakdowns, do not start. 

8. The rule of the knovm evil. People are us6d to working with and 
getting around what they have. 

9. Thi fuji Of the n^^^ . . . the possible beiiefi^ 0f. 
bitter ihfo^m^ll6n» tlier^fore afe mit\9 ^pi^mt In the' pfeMt. 
The costs '116' 1h the fiitQf'e» 
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10. Hypothetical benefits should outweigh estimated costs by at least 
ten to one before everyone concerned starts seeing things. 

Waddlngton (1974), an early pioneer In the field, suggests three more 
rules! 

1. Don't be niggardly with time or money In getting the real facts 
(not opinions, facts). 

2. Heave backward &,id forward from model-making to testing the effects 
of Interim decisions. 

3. It is rarely worth altering existing procedures, which everyone 
concerned is used to, unless the forecast Increase in efficiency 
Is at least 50 percent, preferably 100 percent, to be attained 
In some reasonably short period, before the next alteration becomes 

Reading these rules of thumb is extraordinarily depressing. Waddlngton' s 
experience, as is most of the experience with these methods, has been with v 
the military. Mot only are we educators working with more complex systems 
than the military or business > it seems much less likely that wa vHll be 
provided with the requisite resources in men, machines, or time. 

We are therefore presented with a singularly cruel dilemma^ On the : 
one side, the desperate need for more effective decision making is clear, a 
as is the superiority of the modern management methods over traditional 
apl)roaches. The other horn of the dilemma contains the potential, nay, the 
strong likelihood, that our attempts will be hamstrung through lack of 
resources. Even worse, an attennit to do an MIS without adequate resources 



14 



will almost certainly exacerbate the problems that led to the request for an 
MIS to begin with. 

COMflUNlCATION PROBLEMS 

The difficulties of conmunlcatlon between the practitioner and the 
social scientist are lengendary and well known to all of us. We continue, 
however, to fall In the same old traps. NRRC/P approached CReWS with a 
request to produce an MIS. Their language Included phrases such as "support 
for decision-making," "evaluation," "accountability," "cost-benefit," and 
"cost-effectiveness." Our own predilections were quite compatiblei in 
short, we were excited and Jumped at the chance to develop such a sophisti- 
cated MIS. 

We were very careful to warn the NRRC/P of the (probable) impossibility 
of providing a real cost-benefit capability. We similarly gave solemn and 
frequent warnings about the difficulties and amount of work Involved In the 
whole enterprise. NRRC/P, for their part, continued to repeat their needs 
and pressures on them from the state and the federal governments. 

Despite all these discussions, the parties worked together for a full 
year (conducting the field study primarily) before they really began to 
understand each other. It was mentioned earlier (page 5) that a traumatic 
change in direction (from CReWS perspective) occurred half way through the 
project. In retrospect, it seems incredible that CReWS and NRRC/P could 
have misunderstood each other to such an extent for so long. We would not 
have the cotff*ai6'# It except that history places us 1h vefy 
good company. 
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What can be done to avoid such problems in the future? Probably the 
major safeguard Is to never, never simply take the practitioner's word for 
what he needs. Much more background information and understanding is neces- 
sary and explicit analysis of the situation Is needed to produce that 
understanding. Had we carefully inquired about the concrete problems and 
external pressures that led to the request for an MIS» we would have saved 
a tremendous effort. 

Memlghtnote that uncritical acceptance of the practitioner's word 
for what he needs Is fraught with danger. Given that John Doe educator* 
harassed and overworked, has a problem. He will decide, on some basis, 
that the solution to said problem Is to acquire and utilize a modern wombat. 
His next step, obviously, Is to find an expert in providing wombats, Now 
it makes no difference whether the wombat is a curriculum guide, an adding 
machine, open education, or an HIS. The wombat expert has, almost by defi- 
nition, an ideology which says that wombats are good and necessary for 
education. The expert, therefore, has a very strong temptation to congratu- 
late the practitioner on his keen insight rather than inquiring as to 
whether wombats are truly what Is most needed 1n this particular situation. 

This Is a danger facing everyone of us. Unless we are alert to it 
and firm in our repentance from it, we stand a chance of providing solutions 
for which no real problem exists. We will certainly expend scarce and 
irreplaceable resources on irrelevant projects. At worst, we may further 
exacerbate situations which were already exceedingly difficult, and in the 
process totally discredit the wombdt. The history of education abounds with 
examples of wombats that were adopted with unreasonable expeotatlbni, only 



16 



to be Inevitably followed by disillusionment and disdain for the poor v/ombat. 
All of us who are concerned with the educational enterprise would do well to 
constantly guard against this recurrent syndrome. 

In addition to our own careful Inquiry Into the practitioner's concrete 
problems (as contrasted with his proposed solutions)* It may be helpful to 
secure the analysis of a knowledfleablo but dislntemted third party. 

In any event, the moral is clear that no magic solutions exist, and that 
rushing about without proper preliuifiary analysis, a proper time frame, 
proper understanding and conmitment by all parties to the enterprise, and 
proper resources will be useless at best, Just as the medieval alchemists 
could never chhnQQ base metals into gold,. we cannot change poorly conceptual- 
ized and poorly planned efforts into shining successes through some fortr>ula. 
We do a disservice if we imply that we can. Finally and perhaps most 
importantly, systems tools have no Intrinsic power to Improve qualityi apart 
from the brt)ad perspective of general system theory the tools are largely 
impotent. Thy fact that these tools carry with them an aura of mystery and 
of scientific respectibility makes them attractive to the uninitiated—but 
no less Impotent. If we aspire to be systems experts and to serve education, 
wo are obliged to make ourselves expert in the general theory, to remove the 
aura, and to be prepared for long and arduous labor. 

111. CONCLUSION 

And thus did the family CReWS meet adversity and learn of elixirs and 
come of age in rapid time. CReWS lost its innocence but gained a certain 
Hiiittlty and understanding, and lo, ^t goes forth eyen to this day to do 
Q#1#^aW'to^fltUw^°'t^^^ of ^he systems appro^ach. 
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